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Sir: 

PRELIMINARY AMENDMENT 



In the Claims 



Please add claims 26-31, as follows. 

26. An electronic camera operable in a still 
image mode and in a motion preview mode, the improvement 
wherein 

the still image mode uses a relatively more 
complex digital image processing technique implemented in 
software to produce high quality still images, and the 
motion preview mode uses a relatively more simple digital 



image processing technique implemented in an application 
specific integrated circuit to produce a preview image of 
acceptable quality prior to initiation of the still image 
mode . 

27 . A method for processing images captured by 
an electronic camera that is operable in a still image 
mode and in a motion preview mode, said method comprising 
the steps of: 

processing the images in the still image mode 
by using a relatively more complex digital image 
processing technique that produces a larger number of 
output pixel values than input image pixels to produce 
high quality still images; and 

processing the images in the motion preview 
mode by using a relatively more simple digital image 
processing technique that produces a smaller number of 
output pixel values than input image pixels to produce a 
preview image of acceptable quality. 

28. The method as claimed in claim 27 wherein 
both modes provide color images composed of color pixels . 

29. The method as claimed in claim 28 wherein 
the step of processing the images in the still mode 
includes a color filter array interpolation process that 
produces a larger number of output color pixel values 
than input color pixels. 

30. The method as claimed in claim 28 wherein 
the step of processing the images in the motion preview 
mode maps an array of input color image pixels into a 
smaller array of color display pixels. 
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31. The method as claimed in claim 27 wherein 
the step of processing the images in the still mode is 
implemented in software, and the step of processing the 
images in the motion preview mode is implemented in an 
integrated circuit. 
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AN ELECT RONIC CAMERA HAVING DUAL MODES FOR COMPOSING 
AND CAPTURING STILL IMAGES 

FIELD OF THE INVENTION 

5 

The invention pertains to an electronic still 
camera for composing and capturing still images, and, 
more particularly, to an electronic camera having a 
"motion" mode for previewing a scene and a "still" mode 
10 for capturing a particular image in the scene. 

BACKGROUND OF THE INVENTION 

Consumer camcorders which include the 

15 capability of recording analog motion and/or still 

images on 8mm or VHS videotape have been developed by* a 
number of companies. Motion images are recorded in the 
same manner as in any standard camcorder. These 
cameras include a single chip charge coupled device 

20 (CCD) sensor having a color filter array that provides 
a spatially color-sampled image. To record still 
images, the user pushes a "still capture" button at the 
desired instant. The image obtained from the CCD 
sensor is temporarily stored in a digital memory. The 

25 image is then read from the memory and recorded onto- 
the videotape. Some camcorders include color liquid 
crystal displays (LCD) , which are also spatially color- 
sampled devices. Some are relatively large, for 
example, ranging from approximately 2.5" to 4" in 

30 diagonal. Such a display is used, instead of a normal 
eyepiece viewfinder, to allow the user to properly 
frame the subject and view the images as they are being 
recorded. It is also used to view the recorded images 
as the videotape is played back. 

35 Figure 1A shows a typical color LCD display, 

in which the liquid crystal material is trapped between 
an upper glass plate 1 and a lower glass plate 2. The 
upper plate 1 has a common transparent electrode 3 and 
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an array 4 of color filters surrounded by a black mask 
5. The lower plate 2 includes an array 6 of 
transparent pixel electrodes juxtaposed underneath the 
array 4 of color filters. Individual pixel electrodes 
5 are activated via thin film transistors (TFT) 7 that 
are controlled from a video signal on the source lines 
8 and a scanning signal on the gate line 9 . The LCD 
display includes the usual polarizer layers (not shown) 
on the glass plates 1 and 2, such that activation of 

10 selected transparent pixel electrodes allows light to 
pass through the corresponding color filters and 
reflect to the viewer, thereby creating a color image. 
A typical LCD display such as the Epson LB 2F-BC00, 
manufactured by Seiko-Epson Company, Japan, has about 

15 240 lines of pixels and about 3 00 pixels per line, with 
an image aspect ratio of 4:3. Such an aspect ratio 
allows the entire area of the image obtained from the 
4:3 aspect ratio NTSC format CCD sensor to be displayed 
on the LCD screen, so that the LCD screen composition 

20 will be the same as the image that is recorded by the 

camcorder NTSC format recorder, for later display on an 
NTSC format television display. Note that because the 
LCD has only 240 lines of pixels, the interlaced NTSC 
signal is displayed using a "repeat field" technique, 

25 where both the odd and even fields from the NTSC format 
sensor are displayed using the same lines of pixels on 
the LCD. This LCD, like most commercially available 
LCDs, has "rectangular" pixels, rather than square 
pixels, where the distance between pixels in the 

30 horizontal direction is for example 2/3 the distance in 
the vertical direction. The LCD pixels are overlaid 
with a diagonal RGB stripe pattern as shown in Figure 
IB. 

In camcorders, the processing for both the 
35 still images and the motion images is identical. Such 
processing is normally implemented by hardwired analog 
integrated circuits, although camcorders which use 
digital image processing integrated circuits have been 
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produced. Such camcorders convert the signal from the 
CCD sensor into an NTSC composite or component format 
signal, which is provided to a video recording 
subsystem or a video output jack. The color LCD 
5 display includes circuitry to decode the NTSC composite 
or component signal back into spatially subsampled RGB 
signals to drive the individual RGB pixels on the LCD 
sensor . 

In a system oriented toward still photography, and 
10 in particular a digital still system, it would be 

desirable to avoid the necessity of generating an NTSC 
format signal in order to reduce the complexity of the 
required circuitry. In a totally digital system, that 
is, both the recording and display channels are 
15 digital, it is further desirable to minimize 

incompatibility between the channels. The problem is 
to achieve these objective in an architecture that 
minimizes cost and complexity and maximizes user 
handling . 

20 

SUMMARY OF THE INVENTION 

This problem is solved according to the 
invention by a number of features. In one aspect, the 

25 electronic camera is operable in a still image mode - 
according to a relatively more complex digital image 
processing technique to produce high quality still 
images, and in a motion preview mode according to a 
relatively more simple digital image processing 

30 technique to produce a preview image of acceptable 

quality prior to initiation of the still image mode. 
Such an architecture is particularly adapted to mapping 
an array of color image pixels from a sensor into an 
array of color display pixels on an LCD display 

35 comprising discrete LCD display pixels fewer in number 
than image sensor pixels. In that case, a relatively 
simple digital processing technique combines same- 
colored image pixel signals into a fewer number of 
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intermediate pixels that correspond to the arrangement 
of the color display pixels. 

The advantage of the invention is that the 
two modes can be tailored for a relatively low quality 
5 "motion" mode and a much higher quality "still" mode. 
The motion mode images from the CCD sensor are 
processed by a hardwired digital signal processing 
circuit that generates low resolution, spatially 
subsampled digital image data which can directly drive 

10 the relatively low resolution LCD display. This 
reduces the complexity and clock frequency of the 
required circuitry, compared to generating an NTSC 
format signal, as is normally done in the prior art. 
The still mode image from the CCD sensor is processed 

15 by a general purpose processor (CPU) which executes an 
image processing software program in order to produce a 
high quality digital still image. 

BRIEF DESCRIPTION OF THE DRAWINGS 

20 

The invention will be described in relation 
to the drawings , whereon 

Figures 1A and IB show the structure and 
color filter pattern of a known color liquid crystal 
25 display (LCD) ; 

Figure 2 is a block diagram of an electronic 
camera incorporating dual modes for composing and 
capturing a still image according to the invention; 

Figures 3A and 3B are diagrams of progressive 
30 scan image sensors useful with the camera of Figure 2; 

Figure 4 is a diagram of the Bayer color 
filter geometry for the sensor used with the camera of 
Figure 1 ; 

Figure 5 shows the line timing for the still 
35 mode of operation; 

Figures 6A and 6B show the line timing for 
the preview mode of operation; 
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Figure 7 shows the special line skipping 
pattern used in the preview mode; 

Figure 8 shows further detail of the preview 
mode processing circuit shown in Figure 2; 
5 Figures 9A and 9B show further detail of the 

image statistics processor shown in Figure 2; 

Figure 10 shows one example of still mode 
image processing; 

Figure 11 shows the effect of pixel mapping 
10 from a sensor to an LCD display, each having different 
aspect ratios; and 

Figure 12 shows an enhancement to the block 
diagram of Figure 2 in which a different sensor clock 
frequency is used in each of the dual modes. 

15 

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENT 

A block diagram of a camera incorporating 

20 dual modes of processing according to the invention is 
shown in Figure 2. The camera includes an electronic 
color display, for example, a color liquid crystal 
(LCD) display 10 of the type shown in Figure 1A, and a 
user control section 12 having a number of user control 

25 buttons, including zoom buttons 14, a preview button- 15 
and a capture button 16. To take a still picture, the 
user turns on the camera (using a power switch (not 
shown) , which may be automatically enabled when the 
user depresses the zoom buttons 14 or the preview 

30 button 15, or partially depresses the capture button 
16) . The user composes the picture by depressing the 
"zoom in" or "zoom out" buttons 14, and by adjusting 
the position of the camera, while observing the display 
image. When the user is satisfied with the composition 

35 on the color LCD display 10, the user depresses the 

capture button 16. The camera then captures a single 
still image, firing a flash 18 if necessary when the 
ambient illumination level is low. The still image is 
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focused upon an image sensor 2 0 by a motor driven zoom 
lens 22. The intensity of the image light upon the 
sensor 2 0 is regulated by a motor-driven, variable, 
mechanical aperture 24, while exposure time is 
5 regulated electronically by appropriate clocking of the 
sensor 20. The still image from the image sensor 20 is 
processed and digitally stored on a removable memory 
card 26. 

Control of the sensor is provided by a timing 

10 and control section 27, which is an application 

specific integrated circuit (ASIC) with processing and 
timing functions, for both capture and preview 
operating modes. For instance, the timing and control 
section 27 includes a sensor timing circuit 28 for 

15 controlling the image sensor functions. The sensor 

timing circuit 2 8 provides the signals to enable sensor 
drivers 30, which provide horizontal clocks (HI, H2 ) , 
vertical clocks (VI, V2 ) , as well as a signal FDG for 
activating a drain structure on the sensor 20. The 

20 output of the image sensor 2 0 is amplified and 

processed in an analog gain and sampling (correlated 
double sampling (CDS) ) circuit 32, and converted 'to 
digital form in A/D converter stage 34. The A/D output 
signal is provided to a processor section 35, which 

25 includes a digital processor 3 6 for temporarily storing 
the still images in a DRAM memory 38. The digital 
processor 3 6 then performs image processing on the 
still images, and finally stores the processed images 
on the removable memory card 2 6 via a memory card 

30 interface circuit 40, which may use the PCMCIA 2.0 
standard interface. An EPROM memory 42 is used to 
store the firmware which operates the processor 36. 
The components of the processor 35 are interconnected 
through a data bus 43, which also connects to the 

35 timing and control section 27 and to the card interface 
40. 

The motor-driven zoom lens 22 includes a zoom 
motor 44, a focus motor 46, and an aperture motor 48 
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(all controlled by lens motor drivers 50) . The timing 
and control section 27 further includes a control 
interface 52 connected to the lens motor drivers 5 0 and 
to a flash control circuit 53 via a photosystem 
5 interface block 54, which controls the operation of the 
zoom lens 22 and the flash 18. The lens zoom position 
is controlled by the photosystem interface block 54 
based on position input from the zoom control buttons 
14 through a user status and control section 55. The 

10 focusing, exposure control, and white balance is done 
automatically, as is typically the case in consumer 
camcorders. This is done by computing "image 
statistics" in an image statistics processor 60 in the 
real-time ASIC (timing and control section 27) as 

15 preview images are continuously read out of the image 
sensor 20. The computed values are then used by a 
program implemented in the digital processor 36, which 
decides how to adjust the focus motor, aperture, analog 
gain control, and analog white balance controls via the 

20 control interface 52 and the photosystems interface 54 
on the ASIC timing and control section 27. Although 
the digital processor 3 6 and the control interface 52 
are shown as being within two separate sections, in 
some implementations the same component could be used 

25 to perform both of these functions (as well as other of 
the recited functions) . Sensor image data is passed to 
the processor section 3 5 through a high speed interface 
56 in the timing and control section 27. The sensor 
image data is also directed to the color LCD display 10 

30 through a preview mode processing circuit 58. 

The timing and control section (ASIC) 27 is 
operable in two modes, a relatively low quality 
"motion" mode and a much higher quality "still" mode. 
In the motion mode, images from the sensor 2 0 are 

35 processed by the preview mode processing circuit 58; in 
the still mode, images from the sensor 2 0 are processed 
in the processor 35. The processor 35 is a software- 
driven digital processing system that is slower than 
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the ASIC 27. The preview mode processing circuit 58 is 
a hardwired digital signal processing circuit (part of 
the ASIC 27) that generates low resolution, spatially 
subsampled digital image data which can directly drive 
5 the relatively low resolution color LCD display 10. 

This reduces the complexity and clock frequency of the 
required circuitry, compared to generating an NTSC 
format signal, as is normally done in the prior art. 
The preview mode processing circuit includes a pixel 

10 remapper 62 for mapping the greater number of image 
pixels from the sensor 2 0 into the lesser number of 
display pixels (i.e., corresponding to the array 6 of 
transparent pixel electrodes in Figure 1) in the color 
LCD display 10. The color saturation of the remapped 

15 pixels is then adjusted in a color adjustment circuit 
63 and its output is applied to a multiplexer 64. The 
multiplexer 64 selects image data either from the 
preview mode processing circuit 58, producing a preview 
image, or from the high speed interface 56, which 

20 allows for suitably preprocessed viewing of stored 
images . 

In this camera, the image processing used to 
create the preview mode is done in the timing and 
control ASIC 27, since the processing must be done 

25 rapidly. About 60 images per second are processed in 
preview mode. However, since the image quality of the 
displayed image is limited by the resolution and color 
gamut of the LCD screen of the LCD color display 10, 
there is no need for elaborate . image processing. 

30 Therefore, simple "preview mode" image processing is 
performed in a fixed digital circuit embedded in the 
preview mode processing circuit 58 (which is part of 
the ASIC) . The quality requirements for the still mode 
are much greater, since these images will be downloaded 

35 to a computer, and may be displayed on a high 

resolution CRT display, or printed on a high quality 
thermal printer. Therefore, the digital image 
processing must be more elaborate. By using the 



digital processor 3 6 to implement software procedures 
stored in the firmware memory 42, complex procedures 
can be implemented. These procedures can take several 
seconds to complete, since real-time operation is not 
required. Use of firmware-stored software allows the 
still mode image processing to be upgraded without 
requiring a new ASIC design. In effect, what happens 
is that a relatively less complex digital image 
processing technique is used in the motion preview 
mode, but at a higher data rate, and a relatively more 
complex digital image processing technique is used in 
the still mode, but at a slower data rate. 

Since the update rate, that is, the number of 
images that need to be supplied per unit time, is 
different for the still mode than for the motion mode, 
it is beneficial to use different clock frequencies for 
the different modes of operation. For example, as 
shown in Figure 12, a system oscillator 10 0 produces a 
12 mHz clock frequency for use in the motion mode to 
obtain more updates/second (e.g., 60 images per 
second) , while a divider 102 divides by two to provide 
a 6 mHz clock frequency for the still mode. The lower 
frequency allows more time to accurately position the 
clamp and sample pulses so as to avoid CCD output 
signal transitions. This increases noise immunity in 
the still mode. A multiplexer 104 is enabled by the 
control interface 52 to determine which clock frequency 
is applied to the sensor timing circuit 28. Though not 
specifically shown, the changed timing is also 
communicated to the A/D stage 34 and other timing and 
control circuits . 

The sensor 2 0 is a progressive scan interline 
image sensor having a noninterlaced architecture, as 
shown in more detail in Figure 3A. The sensor 
comprises a two-dimensional array of photosites 66, 
e.g. photodiodes, arranged in rows and columns of image 
pixels, a plurality of vertical registers 68 adjacent 
photosite columns for transferring rows of image pixel 
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charge from the photosites 66 to a horizontal register 
70 for readout responsive to clock signals from the 
sensor drivers 30, and a charge drain (specifically, a 
fast dump structure 72) interposed between the output 
5 of the vertical registers 68 and the horizontal 

register 7 0 for eliminating complete rows of image 
pixels at a time from the image sensor 20. A preferred 
image sensor is the Kodak model CCD KAI-0400CM image 
sensor, which has approximately 512 active lines with 

10 approximately 768 active pixels per line and an image 
aspect ratio of 3:2. This sensor is described in a 
Performance Specification document available from 
Eastman Kodak Company, Rochester, New York. Each pixel 
is 9 microns "square", since both the vertical and 

15 horizontal distances between the centers of adjacent 
pixels are 9 microns. The 3:2 image aspect ratio of 
the CCD sensor, although wider than the 4:3 aspect 
ratio of the display, is considered to be a preferred 
aspect ratio for still photography, in that the 

20 standard 35mm film format, and standard 4R (4" x 6") 

prints also have a 3:2 image aspect ratio. The sensor 
uses a color filter array pattern known as the "Bayer 
checkerboard" pattern, described in U.S. patent 
3,971,065, which is shown in Figure 4. Such a color 

25 filter array is characterized by a mosaic pattern in_ 
which the filter colors alternate in both line and 
column directions. In the normal operating mode, all 
of the image pixels on the sensor are transferred as 
color image pixels to the horizontal register 70, which 

30 delivers a stream of color pixel signals to the analog 
gain and CDS circuit 32 (see Figure 2) . The color 
pixel signals are subsequently converted to digital 
pixel signals in the A/D converter 34. 

The sensor 2 0 uses a progressive scan readout 

35 method, which allows the entire image to be read out in 
a single scan. The accumulated or integrated charge 
for the photodiodes comprising the pixels 66 is 
transported from the photosites to light protected 
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vertical (parallel) registers 68 by applying a large 
positive voltage to the phase-one vertical clock (VI) . 
This reads out every row, or line, into the vertical 
registers 68. The image pixel charge is then 
5 transported from the vertical registers 68 to the 
horizontal register 70 by two-phase clocking of the 
vertical clocks (VI, V2) . Between the vertical and 
horizontal registers is the fast dump structure 72, 
which is further described in the Performance 

10 Specification document for the KAI-0400CM sensor. By 
setting a suitable positive potential on a fast dump 
gate line FDG, charge from the row of pixel values 
currently adjacent to the fast dump structure 72 is 
transferred from the CCD directly into the sensor 

15 substrate 74 rather than to the horizontal register 70. 
This dump, or line clear, is accomplished during the 
vertical-to-horizontal transfer time. When properly 
controlled by the sensor timing circuit 28, the fast 
dump structure 72 allows lines of charge to be 

20 eliminated. 

The timing and control section 27 operates 
the electronic camera shown in Figure 2 in the two 
aforementioned modes, including a first "still" mode 
wherein all rows of image pixel charge corresponding to 

25 each line are progressively read out through the 

horizontal register 70 during a single scan, and a 
second "motion" mode wherein some of the rows of image 
pixel charge corresponding to some lines are eliminated 
through the fast dump structure 72 prior to readout. 

30 As applied to the embodiment of Figure 2, the first 

mode corresponds to a high quality still imaging mode 
while the second mode corresponds to a special "line 
skipping" mode for driving the color LCD display 10. 
In the second mode, the timing and control section 27 

35 controls the fast dump structure 72 to eliminate two or 
more consecutive lines of image charge from the image 
sensor 20 for every one or more lines of image charge 
that are transferred to the horizontal register 7 0 for 
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readout, thereby generating a pattern of lines (shown 
in Figure 7) suitable for driving the LCD display in a 
"repeat field" mode. An appropriate video signal which 
displays the entire 3:2 aspect ratio sensor image on 
5 the 4:3 aspect ratio LCD, without introducing geometric 
distortion, is generated by alternately eliminating two 
or four consecutive lines of image charge for every 
pair of lines of image charge that are transferred to 
the horizontal register 70. 

10 The sensor timing circuit 28 is controlled by 

the control interface 52 to provide the clock signals 
VI, V2, HI, H2, and the gate signal FDG according to 
the two modes of operation. The timing signals for the 
first mode are shown in Figure 5; those for the second 

15 mode are shown in Figure 6a and 6b. The two-phase 

cycling of signals VI and V2 control the transfer of 
lines of image pixel charge from the vertical registers 
68 to the horizontal register 70. The two-phase 
cycling of signals HI and H2 control the transfer of a 

20 stream of color pixel signals from the horizontal 

register 70 to subsequent circuits in the camera. The 
level of the signal FDG determines whether the image 
charge is dumped to the substrate 74 or transferred to 
the horizontal register 70. When the sensor 20 is 

25 clocked using the first timing mode shown in Figure 5 
for all lines of the sensor, all lines of the sensor 
are clocked out, one after the other, through the 
horizontal register 70, processed in subsequent camera 
circuitry, and stored in the removable memory 26. This 

30 timing mode provides a high quality progressive scan 

still image, but may take 1/30 second or longer to read 
out the still image. Such timing, however, is 
acceptable for still mode usage, and, as mentioned 
before, does not require unusually high speed 

35 components and, indeed, may benefit from a lower speed 
clock. 

To provide an image to the color LCD display 
10, a lower resolution image is suitable, but the 
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update rate must be sufficient to provide good motion 
rendition and eliminate display flicker. Moreover, the 
sensor 20 includes the aforementioned array of color 
filters arranged in a particular mosaic color pattern 
5 (e.g., the checkerboard Bayer pattern of Fig. 3), and 
the lines of image charge that are transferred to the 
horizontal register 7 0 should preserve that particular 
color pattern in the pattern of lines that are 
generated for the line-skipping readout. To provide 

10 this kind of image, the sensor is read out in the 
second mode as shown in Figure 7, using the timing 
shown in Figures 6A and 6B. As shown in Figure 6A, the 
first two lines (1 and 2) are read out as in the normal 
mode. These provide a green-red and a blue-green line. 

15 The next two lines (3 and 4) are eliminated by turning 
on the fast dump structure 72 during the time that 
these lines are transferred past the fast dump 
structure 72. Next, as shown in Figure 6B, lines 5 and 
6 are read out normally, and then lines 7, 8, 9, and 10 

20 are eliminated. Next, the Figure 6A timing is used to 
read out lines 11 and 12, while eliminating lines 13 
and 14, and then the Figure 6B timing is used to read 
out lines 15 and 16, while eliminating lines 17-20. 
This process proceeds for the entire image readout 

25 period, during which 102 pairs of lines are read out-, 
and 154 pairs of lines are eliminated. 

This special "line skipping" readout mode, as 
shown in Figure 7, allows the sensor 3:2 aspect ratio 
image to be fully displayed on a 4:3 aspect ratio LCD 

30 without "geometric distortion", that is, without 

stretching the image vertically, and without cropping 
off the horizontal edges of the image from the image 
sensor. This allows the LCD to properly show the 
entire 3:2 aspect ratio image captured by the sensor, 

35 so that an image can be properly composed. 

As the 512 lines of the CCD imager are read 
out using the special "line skipping" mode, they are 
displayed using only 204 out of the approximately 24 0 
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LCD lines of pixels. The remaining approximately 36 
lines can either be masked behind a bezel, so that they 
are not visible, or preferably may be used to display 
status information, such as the time-of -day, image 
5 number, or a "push-button menu" for the user buttons. 
Figure 11 shows a useful application of such 
conversions. A sensor having a 3:2 aspect ratio is 
shown mapped into an image area 90 of an LCD display 
screen 92 having a 4:3 aspect ratio. A proportional 

10 remapping leaves a status area 94 available for other 
purposes, specifically to show text indicating the 
function of a set of reconf igurable control buttons 96 
in the control section 12 . The function of the buttons 
is specified by the user status and control section 55 

15 (Figure 2). This status information graphics data can 
be supplied by the digital processor to the LCD 10 via 
high speed interface 56, when the MUX 64 is controlled 
so as to use the digital data from interface 56, rather 
than from circuit 58, for supplying data to the final 

20 approximately 32 lines of the display 10. 

The "line skipping" readout causes some minor 
vertical sampling artifacts, but these are not 
noticeable in the small LCD displays. The pixels 
output for the sensor 2 0 in line skipping mode continue 

25 to have the Bayer-type color filter repeating pattern, 
so that they can be processed using processing 
techniques designed for the Bayer pattern. 

The processing complexity of the camera of 
Figure 2 is considerably simplified by directly mapping 

30 the RGB sensor pixels 66 to the RGB pixels of the 

display 10. The easiest way to do this for the image 
sensor 20 (512 lines x 768 pixels and 3:2 aspect ratio) 
is to have an LCD display with 512 x 768 pixels and the 
same aspect ratio and color filter pattern. However, 

35 this would be a custom,, costly LCD. Instead, LCDs have 
fewer display pixels than image pixels on the image 
sensor, normally have a 4:3 image aspect ratio, and use 
the diagonally striped RGB pattern shown in Figure IB. 
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In this discussion, an LCD pixel array of about 240 
lines x 312 pixels per line is assumed. 

Therefore, the sensor pixels are processed in 
a "pixel mapping circuit", such as the LCD pixel 
5 remapper 62 . A block diagram of this circuit is shown 
in Figure 8. Note that there are 768/2 = 3 84 green or 
red/blue pixels per line on the sensor (see Figure 4) . 
There are about 3 00/3 = 100 green, red, or blue pixels 
per line on the LCD (see Figure IB) . Thus, there are 

10 approximately 1/4 as many LCD pixels per line (per 

color) as there are sensor pixels per line. Therefore, 
the basic plan is to combine same-colored image pixels 
into a fewer number of intermediate, combination pixel 
signals that are then mapped into the color display 

15 pixels. For instance, a simple "pixel mapping" circuit 
maps four green sensor pixel signals into one green LCD 
pixel for one line by summing two green sensor values, 
spaced apart by 4 CCD pixel positions, in the green 
pixel summer 7 6 and dividing by two via bit-shift 

20 wiring. The necessary delay is provided by the 

registers 82 clocked at one fourth the pixel rate, 
further delayed by one pixel clock. It also maps four 
red sensor pixels into one LCD pixel in the same manner 
(using the red/blue summer 78), and also stores this 

25 value in a 100 pixel FIFO 80. The FIFO 80 compensates 
for the fact that the sensor has line sequential red 
and blue pixels, by supplying blue pixels on the red 
sensor lines, and red pixels on the blue sensor lines. 
Four pixel delays are provided by the registers 82 ' 

30 clocked at one fourth pixel rate (CLK/4) . The mapping 
process is basically, therefore, a process which, in 
its simplest form, involves averaging of signals to 
produce a smaller number of output color pixels than 
input color pixels. (The CFA interpolation algorithm 

35 discussed in reference to Figure 10, on the other hand, 
produces a larger number of "output" color pixels than 
input color pixels.) Alternate groups of 2 or 4 lines 
of sensor values are discarded during preview mode by 
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using the fast dump gate shown in Figure 3A, as 
described in connection with the "line skipping" mode. 
This allows the sensor readout time to be decreased by 
more than 1/2 during the preview mode. 

Another feature of the design is that by 
removing the NTSC rate driving circuitry from the color 
LCD display 10, the active matrix LCD can be updated at 
a slower frequency than is normally used. This reduces 
the cost and power consumption of the LCD driver 
circuits (not shown) . For example, the LCD can be 
updated at 3 0 Hz (provided the LCD active matrix 
display is designed so as not to exhibit noticeable 
flicker at this update rate), instead of 60 Hz. 

Once the LCD pixel values are calculated, the 
LCD color adjust circuit 63 increases the color 
saturation of the image by forming R-G and B-G color 
difference signals, and adding or subtracting a 
fraction of these signals in an array 84 of adders and 
subtracters from the original RGB signals in order to 
increase the color saturation of the displayed image. 
This circuit performs a similar function as a 3x3 color 
matrix, but uses less hardware and provides less color 
accuracy. The color accuracy is not critical for the 
LCD display, however, due to the limited color 
reproduction quality of such displays. The color 
reproduction of the still image is much more important, 
and is done with a more complicated and precise color 
correction method with the stored firmware in EPROM 42 . 

Figures 9A and 9B show the processing in the 
image statistics processor 60, which computes real-time 
values for both still and motion image capture, all 
during the preview mode. These values include 24 R, G, 
and B averages used to set white balance and exposure, 
and 6 average high frequency "detail" values used to 
set focus. The 24 RGB averages for white balance are 
calculated for a group of 4 x 6 rectangular image 
regions, for each of R, G, and B; a block diagram of 
the calculation in one color is shown in Figure 9A. 



-17- 



The 6 average "detail" values for focusing are 
calculated for green pixels only by accumulating the 
absolute value of the differences between nearby green 
values; a block diagram of this calculation is shown in 
Figure 9B. These values are computed for each preview 
image and downloaded to the processor 36. The 
processor 3 6 implements a firmware stored procedure 
which determines the optimum exposure parameters 
{exposure time, f /stop, and analog gain) , white balance 
settings, and lens focus setting. For the still mode, 
the processor 3 6 also decides, based on the last 
preview images, whether to fire the flash, and 
determines the optimum exposure parameters, white 
balance settings, and lens focus setting for the still 
mode . 

Once the still image is captured, the digital 
processor 3 6 implements the stored firmware procedures 
to process and store the still image. Figure 10 shows 
a diagram of one possible still image processing 
method. The CFA interpolation diagram may include the 
green interpolation method described in U.S. patent 
application USSN 085,520, filed June 30, 1993, in the 
name of the same assignee as the present application, 
and the chrominance interpolation method described in 
U.S. patent 4,642,67 8, both of which are incorporated 
herein by reference. The color matrix, tone 
correction, and edge enhancement steps may be similar 
to those described in U.S. patent 5 , 189 , 511 , also 
incorporated herein by reference. The image 
compression method may be the JPEG standard compression 
technique . 

The foregoing description envisions taking a 
single still picture following the motion preview mode.' 
The camera can also optionally capture "bursts" of high 
quality still images into the DRAM memory during the 
"still" mode, which are then processed as shown in 
Figure 10. Owing to the flash recharge time and other 
limitations of the "burst" mode, the "burst" mode could 
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utilize different exposure parameters (exposure time, 
aperture, analog gain, flash, and digital processing) 
than either the motion or the single still mode. 

The invention has been described with 
5 reference to a preferred embodiment. However, it will 
be appreciated that variations and modifications can be 
effected by a person of ordinary skill in the art 
without departing from the scope of the invention. For 
instance, Figure 3B shows a progressive scan sensor 

10 with two readout registers 86 and 88 (which corresponds 
to the Performance Specification document for the Kodak 
KAI-031CM image sensor; the preferred embodiment of 
Figure 3A simply uses but one register) . The purpose 
is to eliminate the FIFO line delay 80 in the LCD pixel 

15 remapper 62. The pairs of lines read out by the 
registers include both a green/red line and a 
blue/green line. Therefore, by adding an analog 
multiplexer between the outputs of the two channels and 
the analog gain and CDS block 32, which is switched at 

20 1/2 the sensor horizontal clock rate, it is possible to 
obtain a GRBG sequence of sensor data values at the 
output of the A/D stage 34. The LCD pixel remapper 62 
can then be designed to map from the CCD sensor color 
pixel pattern to the required RGBRGB LCD pixel pattern, 

25 without using a line delay. Since two CCD lines are 
read out in parallel, for each LCD line, fewer lines 
are eliminated via the fast dump gate than for the 
single-register sensor shown in Figure 3A. 

In particular, if the dual register CCD in 

30 Figure 3B had 512 x 768 square pixels with a 3:2 aspect 
ratio, and the LCD had approximately 240 display lines 
and a 4:3 aspect ratio, the CCD readout would involve 
fast dumping one pair of lines for every four pairs of 
lines read out from the CCD sensor. The pixel readout 

35 procedure for the horizontal registers can then be 
varied depending on the mode of operation: both 
registers are used for the motion imaging mode and one 
register is used for the still imaging mode. 
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Furthermore, although the Bayer pattern was described, 
other mosaic-type filter patterns could be used to 
advantage, for example, complementary patterns 
involving cyan, magenta, yellow and green filters. The 
processing for the LCD pixel remapper 62 and the LCD 
color adjust circuit 63 would be accordingly modified 
to account for the different color arrangement. 
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color LCD display 


12 


control section 


14 


zoom buttons 
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preview button 
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capture button 


18 


flash 
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30 


sensor drivers 
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A/D converter 
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card interface 
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zoom motor 


46 
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48 
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55 


user status and control section 
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56 high speed interface 

58 preview mode processing circuit 

60 image statistics processor 

62 pixel remapper 

5 63 color adjustment circuit 

64 . multiplexer 

66 pixel 

68 vertical readout register 

70 horizontal register 

10 72 fast dump structure 

74 sensor substrate 

7 6 green pixel summer 

7 8 red/blue summer 

80 FIFO 

15 82 two pixel delay registers 

84 array 

86 first readout register 

88 second readout register 

90 image area 

20 92 LCD display screen 

94 status area 

96 reconf igurable buttons 

100 oscillator 

102 divider 

25 104 multiplexer 
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CIAIMS ; 

1. An electronic camera operable in a still 
image mode and in a motion preview mode, the 

5 improvement wherein 

the still image mode uses a relatively more 
complex digital image processing technique to produce 
high quality still images, and the motion preview mode 
uses a relatively more simple digital image processing 
10 technique to produce a preview image of acceptable 

quality prior to initiation of the still image mode. 

2 . An electronic camera as claimed in claim 
1 wherein both modes operate on color images . 

3 . An electronic camera as claimed in claim 
15 2 further including an image sensor having an array of 

color image pixels and a display having a smaller array 
of color display pixels, and wherein the image 
processing technique used during the motion preview 
mode maps the array of color image pixels into the 
20 smaller array of color display pixels. 

4. An electronic camera as claimed in claim 
3 wherein the image processing technique used during 
the motion preview mode also modifies the saturation of 
the color image pixels . 

25 5. An electronic camera as claimed in claim 

3 wherein the display is a liquid crystal display (LCD) 
comprising discrete LCD display pixels, and the image 
processing method used during the motion preview mode 
maps . more than one image pixel from the sensor to 

30 generate each value for individual LCD display pixels. 

6. An electronic camera as claimed in claim 
1 wherein the more complex image processing technique 
implemented in the still image mode is implemented in 
software, and more simple image processing technique 

35 implemented in the motion preview mode is implemented 
in an application specific integrated circuit. 
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7. An electronic camera operable in a still 
image mode and in a motion preview mode, said 
electronic camera comprising: 

an image sensor including a two-dimensional 
array of image pixels covered by a mosaic pattern of 
color filters, said image sensor providing a stream of 
color pixel signals at an output thereof suitable for 
still imaging ; 

an analog-to-digital stage for converting the 
color pixel signals to digital pixel signals; 

a color display comprising a discrete two- 
dimensional arrangement of color display pixels, said 
color display having fewer color display pixels than 
the number of image pixels on the image sensor; and 

a preview mode processor for mapping the 
digital image pixel signals into at least a viewable 
portion of the color display pixels by combining same- 
colored image pixels into a fewer number of 
intermediate pixels that correspond to the arrangement 
of the color display pixels. 

8. An electronic camera as claimed in claim 

7 wherein the mosaic pattern of color filters covering 
the image sensor is different than the arrangement of 
color display pixels. 

9 . An electronic camera as claimed in claim 

8 wherein said mosaic pattern of color filters is the 
following pattern 

R G R G 
G B G B 
R G R G 
G B G B 

and wherein the preview mode processor combines the 
digital pixel signals into a fewer number of RGB pixels 
by averaging two or more of the same-colored image 
pixels in each line to provide averaged display pixels. 

10. An electronic camera as claimed in claim 

9 wherein the preview mode processor further modifies 
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the saturation of the averaged display pixels by adding 
or subtracting portions of one or more of the other 
colors to each averaged display pixel. 

11. An electronic camera as claimed in claim 
5 7 wherein the image sensor and the viewable portion of 
the color display have the same aspect ratios such that 
substantially all of the image pixel signals can be 
mapped into substantially all of the color display 
pixels . 

10 12. An electronic camera as claimed in claim 

7 wherein the image sensor and the color display have 
different aspect ratios such that substantially all of 
the image pixel signals are mapped into the viewable 
portion of the color display pixels, leaving another 

15 portion of the color display pixels available for non- 
imaging use. 

13 . An electronic camera operable in a still 
image mode and in a motion preview mode, said 
electronic camera comprising: 

20 an image sensor comprising a two-dimensional 

color filter array, a two-dimensional array of image 
pixels arranged in rows and columns with respect to the 
color filter array, and a horizontal register for 
outputting rows of color pixel signals; 

25 an analog-to-digital stage for converting the 

color pixel signals to digital pixel signals; 

a color display comprising a specific two- 
dimensional color pattern of display pixels arranged in 
rows and columns, having substantially fewer rows and 

30 fewer columns than the image sensor; 

a still image processor for processing the 
digital pixel signals obtained during the still image 
mode , and 

a preview mode processor for mapping the 
35 digital pixel signals obtained during the motion 

preview mode into the display pixels according to a 
procedure that digitally maps the digital pixel signals 
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corresponding to the color filter array of the sensor 
into the color pattern of the color display. 

14 . An electronic camera as claimed in claim 
13 wherein the preview mode processor further modifies 

5 the saturation of the digital pixel signals. 

15. An electronic camera as claimed in claim 
13 wherein the still image processor utilizes a 
software procedure for processing the digital pixel 
signals and the preview mode processor includes a fixed 

10 digital circuit embedded in an application specific 
integrated circuit (ASIC) for processing the digital 
pixel signals. 

16. An electronic camera as claimed in claim 
15 wherein the ASIC processes the digital pixel signals 

15 more rapidly than the software procedure. 

17 . An electronic camera as claimed in claim 
13 wherein the color filter array covering the image 
sensor is different than the color pattern of display 
pixels . 

20 18. An electronic camera as claimed in claim 

17 wherein said color filter array has the following 
pattern 

R G R G 

25 G- B G B 

R G R G - 
G B G B 

and wherein the preview mode processor combines the 
30 digital pixel signals into a fewer number of RGB pixel 

signals by averaging two or more of the same-colored 

image pixel signals in each line to provide averaged 

RGB pixel signals. 

19. An electronic camera operable in a still 
35 image mode and in a motion preview mode, said 

electronic camera comprising: 

an image sensor comprising a two-dimensional 

array of image pixels arranged in rows and columns, and 
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a horizontal register responsive to clock signals for 
outputting rows of image pixel signals; 

an analog-to-digital converter for converting 
the output image pixel signals into digital image pixel 
5 signals; 

a still image processor for processing the 
digital image pixel signals obtained during the still 
image mode; 

a motion preview processor for processing the 
10 digital image pixel signals obtained during the motion 
preview mode; and 

a timing generator for operating the image 
sensor according to respectively different clock signal 
frequencies depending upon which operating mode is 
15 being used, 

20. An electronic camera as claimed in claim 
19 wherein the clock frequency for operating the image 
sensor in the motion preview mode is higher than the 
clock frequency for operating the image sensor in the 

20 still image mode. 

21. An electronic camera operable in a still 
image mode and in a motion image mode, said electronic 
camera comprising: 

an image sensor including a two-dimensional 
25 array of image pixels and a horizontal readout section 
for providing an output stream of image pixel signals; 

an analog-to-digital stage for converting the 
output image pixel signals into digital image pixel 
signals; and 

30 a timing and control section for driving the 

image sensor to provide the output stream of image 
pixel signals according to a pixel readout procedure 
involving clock signals, said pixel readout procedure 
being different for the two modes. 

35 22. An electronic camera as claimed in claim 

21 wherein the pixel readout procedure comprises 
separate clock signal rates for the still image mode 
and the motion image mode. 
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23. An electronic camera as claimed in claim 
22 wherein the clock signal rate for the still image 
mode is slower than for the motion image mode. 

24. An electronic camera as claimed in claim 
5 21 wherein the horizontal readout section comprises two 

horizontal readout registers and the pixel readout 
procedure comprises use of one horizontal register for 
the still image mode and the use of both horizontal 
registers for the motion image mode. 
0 25. An electronic camera as claimed in claim 

1 wherein the still image mode includes the capture of 
a single image or the capture of a burst of images. 
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AN ELECTRONIC CAMERA HAVING DUAL MODES FOR COMPOSING 
AND CAPTURING STILL IMAGES 

ABSTRACT 

5 

An electronic camera uses a relatively more 
complex digital image processing technique in a still 
image mode to produce high quality still images, and a 
relatively more simple image processing technique in a 

10 motion preview mode to produce preview images of 

acceptable quality prior to initiation of the still 
image mode. The more complex digital technique is done 
in software in a general purpose processor section 35, 
while the more simple digital technique is implemented 

15 in a fixed digital circuit in an application specific 

integrated circuit 27, which also implements timing and 
control functions. The motion preview mode uses a 
shorter image readout period than the still mode and 
further involves mapping image sensor pixels into a 

20 fewer number of color display pixels on a color LCD 

display 10. The mapping further converts color pixel 
signals from a mosaic array into a different color 
pattern on the color LCD display 10. 
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